The Maximum Entropy on the Mean Method, Noise and Sensitivity

In this paper we address the problem of building convenient criteria to solve linear and noisy inverse problems of the form y = Ax + n. Our approach is based on the speci cation of constraints on the solution x through its belonging to a given convex set C. The solution is chosen as the mean of the distribution which is the closest to a reference measure on C with respect to the Kullback divergence, or cross-entropy. This is therefore called the Maximum Entropy on the Mean Method (memm). This problem is shown to be equivalent to the convex one x = argminxF(x) submitted to y = Ax (in the noiseless case). Many classical criteria are found to be particular solutions with di erent reference measures . But except for some measures, these primal criteria have no explicit expression. Nevertheless, taking advantage of a dual formulation of the problem, the memm enables us to compute a solution in such cases. This indicates that such criteria could hardly have been derived without the memm. In order to integrate the presence of additive noise in the memm scheme, the object and noise are searched simultaneously for in an appropriate convex C. The memm then gives a criterion of the form x = argminx F(x) + G(y Ax), where F and G are convex, without constraints. The functional G is related to the prior distribution of noise, and may be used to account for speci c noise distributions. Using the regularity of the criterion, the sensitivity of the solution to variations of the data is also derived. 1. Problem statement In many applications, one often faces the inverse problem y = Ax + n which consists in estimating a vector x 2 IR from an indirect and noisy observation vector y. The observation matrix A is supposed to be known, together with some statistical characteristics of the noise n. When the observation matrix A is either not regular or ill-conditioned the problem is ill-posed and one has to complete the data with an a priori knowledge or constraints on the solution in order to select a physically-acceptable solution. Such information may be given in the form of the convex constraint x 2 C; (1) where C is a convex set. Examples of this situation are plentiful, let us only cite the problem of imaging positive intensity distributions, which arises in spectral analysis, astronomy, spectrometry, etc: : : In other speci c problems, such as crystallography or tomography, lower and upper bounds on the image are known, and have to be taken into account in the reconstruction process. Such constraints may be speci ed by the belonging of the object to the convex set C (where the bounds ak and bk are given) and include the positivity constraint as a special case, C = f x 2 IR= xk 2 ]ak; bk[ ; k = 1::Ng: (2) 2 J.-F Bercher, G. Le Besnerais and G. Demoment 2. Methods for solving linear inverse problems In the case of an ill-posed problem, the generalized inverse solution is unsatisfactory because of the dramatic ampli cation of any observation noise. Quadratic regularization makes possible to get rid of ill-posedness e ects, but it leads to linear estimates, and therefore cannot provide any guarantee with respect to the support constraint (2). Possible answers are given with set theoretic estimation (for a review see [1]) and projection onto convex sets algorithms. Although good reconstructions can be obtained, they are often computationally expensive and do not lead to a unique and well-de ned solution. Other approaches use regularized criteria, which are usually written as a compound criterion made of two terms, one which enforces some delity of the solution to the data, the other which ensures that some desirable properties are met. Such regularized criteria will be noted under the generic form J (x) = F(x) + G(y Ax) 0: (3) Many of these regularized criteria may be interpreted in a Bayesian setting. Indeed, if the functionals F and x 7! G(y Ax) are respectively a log-prior and a log-likelihood, then the minimization of J provides the maximum a posteriori (MAP) estimator. However, in a given problem, the ab initio choice of a good model is a di cult task, for which there is no general answer (see [6] for a discussion of the subject). Such situations are encountered when the only a priori knowledge is a convex constraint such as (1). Nevertheless, useful methods have been found in those cases: for instance, when reconstructing object with positivity as the only pre-requisite, several thought processes have lead di erent authors to the conclusion that the maximum entropy reconstruction method could be a useful answer. It consists in the optimization of a regularized criterion of the form (3) with